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PREFACE 



This is a brief summary of a report of a two-year study of the 
National Science Foundation's Comprehensive Assistance to Undergraduate 
Science Education (CAUSE) program. The study and its conclusions, 
although supported by quantitative data, rest heavily on the qualitative 
analysis of a series of site visits and in-depth case studies of indivi- 
dual CAUSE projects. This analysis occurred at multiple levels through- 
out the course of the study — during the visits themselves; during the 
process of writing > rav/r i ting and reviewing site visit and case study 
reports; during formal and informal meetings of the project staff; and 
during the writing of the final report which assimilates the analyses of 
both the qualitative and quantitative aspects of the study. Because this 
analytic process is subjective by its very nature, we have tried to open 
it to review through the three volumes of the evaluation report. The 
reader is referred to the site visit reports and the case studies in par- 
ticular for an understanding of how we came to our conclusions as well as 
how the CAUSE program operates in context. The site visit reports and 
the case studies are an integral part of the study and should not be con- 
sidered as supplementary appendices. 
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OVERVIEW OF THE EVALUATION REPORT 



Volume I: Overview and Findings 



Chapter One, Overview of the Evaluation , contains a description and 
the history of the CAUSE program and a brief introduction to the evalu- 
ation of CAUSE. 

Chapter Two, Methodology of the Evaluation , contains a description 
of the evaluation issues, the framework of the evaluation, data-gathering 
methods and activities of the case studies, site visits, survey, and 
analysis of proposals. 

Chapter Three, Conclusions , provides conclusions for each of the six 
issues of the evaluation based on the analysis of data from all the 
sources (survey, proposal review, site visits, case studies). 

Chapter Four, Recommendations to NSF , presents a discussion based on 
conclusions reached in Chapter Three and provides recommendations to NSF 
for program modifications. 



Volume II: Visits to 25 CAUSE Projects 



Chapter One, Analysis of Findings from Visits to CAUSE Projects , 
. presents a descriptive assimilation of the issues based on all the site 
visit reports and all the case studies. 

Chapter Two, Case Studies of Eight CAUSE Projects , consists of eight 
case studies which describe in-depth the project, visits made over the 
period of a year and the issues as they emerged over the_course of a 
detailed study. 

Chapter Three, Site Visits to Seventeen CAUSE Projects , consists of 
17 site visit reports which describe the project, the methodology, and 
the issues as they emerged at each site. 



Volume III: Broad View of CAUSE Projects 



Chapter One, The Survey of CAUSE Project Directors , presents an over- 
view of the survey; a discussion of the development of both surveys and 
the analysis procedures; the conclusions, issue by issue, as they emerged 
from both surveys taken together; and a detailed analysis of each survey, 
item by item, together with tabled data. 

Chapter Two, Content Analysis of Funded Proposals , discusses the 
techniques employed in analyzing the 273 proposals funded between 1976 and 
1979 and presents an analysis of the results of those procedures. 
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OVERVIEW OF THE EVALUATION 



Overview of the CAUSE Program 



According to the 1979 program announcement the Comprehensive 
Assistance to Undergraduate Science Education (CAUSE) program 
encourages the improvement in quality and effectiveness of under- 
graduate science education in institutions of higher education. 
The primary objectives of the CAUSE program are to: (1) strengthen 
the resources for undergraduate science education components of 
2-year and 4-year colleges and universities; (2) improve the quality 
of science instruction at the undergraduate level; and (3) enhance 
the capability of institutions for self-assessment, management, and 
evaluation of their science programs. 

Any nonprofit, degree granting institution or consortium of 
institutions in the United States is eligible to submit a proposal. 
Program objectives are to be achieved through projects identifying 
and meeting local science education needs. CAUSE is intended to 
provide for comprehensive re v i tal i za t i on of an instructional pro- 
gram either within or across departments. Funds cannot be requested 
for major construction or additional faculty except as required 
for improving an instructional program. Support is intended to 
result in improvements which will continue beyond the actual funding 
period. Grants may be requested for up to $250,000 over a 3-year 
period. The proposing institution must provide at least one-third 
of the total project costs. 

Any physical or social science discipline is eligible for 
CAUSE monies as are interdisciplinary efforts in the sciences. 
Each institution is allowed to submit only one proposal. This 
necessitates at the institutional level the development of some 
procedure, formal or informal, for determining which department or 
group of departments will submit a proposal to the CAUSE program. 

Proposals are grouped for review into categories of 2-year 
colleges, baccalaureate-granting institutions, Ph . D . -g ran t i n g 
institutions and consortia. A consortium may be formed for the 
sole purpose of planning a project which will benefit several 
institutions. A system of peer review has been established for 
assessing the proposals. The majority of peer reviewers assigned 
to read proposals from baccalaureate-granting institutions are 
themselves members of such institutions. The same is true for the 
other institutional types. Similarly, biologists tend to be 
assigned to read biology proposals, physicists to read physics pro- 
posals, and so on . 

The CAUSE program was founded by an act of Congress in 1975 
and awarded its first 59 grants on June 18, 1976. It has just 
completed its fourth year. 
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History and Background of CAUSE 

Origins 

CAUSE is a fairly young program; it is the youngest of the 
three NSF programs now aimed at improving college science programs 
(the other two are ISEP, Instructional Science Equipment Program 
and LOCI, Local Course Improvement Program). However, CAUSE has 
its origins in the College Science Improvement Program (COSIP) 
which began in 1967. This was a program whose goal was to broaden 
the availability of superior science programs in all regions of 
the United States. Specifically, COSIP was responsive to the needs 
of 2-year, 4-year and minority institutions to improve their 
science programs' at a time when most grfnt programs were aimed at 
the great research universities. In 1973, however, cuts in NSF's 
budget forced a discontinuation of the COSIP program. 

Congressional Mandate for CAUSE 

When the National Science Foundation presented its Fiscal 
Year 1976 budget to Congress in the spring of 1975, no mention was 
made of any broad undergraduate science program. CAUSE resulted 
from hearings on the NSF budget by the Subcommittee on Science, 
Research and Technology, U. S. House of Representa t i ves . The 
immediate cause was public testimony in which the need for support 
for the improvement of undergraduate science education was stressed* 
The 1976 NSF Authorization Act, Public Law 94-86; contains this 
section pertaining to CAUSE: 

The National Science Foundation is authorized and 
directed to conduct a Comprehensive Assistance to 
Undergraduate Science Education program, referred 
to hereinafter as CAUSE. CAUSE shall have the 
purpose of strengthening the science education 
capabilities of predominantly undergraduate edu- 
cational institutions and departments or groups 
of departments thereof through awards to four-year 
colleges, two-year colleges, to the undergraduate 
component of advance degree institutions, and to 
groups of such institutions, . . 



Purposes and Objectives of the Program 

The Committee Conference of the two Houses reviewed the pur- 
pose of CAUSE and its potential role within the overall NSF science 
educati on program . The Conference Report includes this statement 
describing congressional intent as it related to CAUSE. 

The CAUSE program, i ni tia ted by the commi ttee for 
the coming fiscal year, will provide a means of 
strengthen! ng undergraduate science education in 



ERLC 



,9 



3 



the Nation's colleges. This program will provide 
specialized science teaching equipment, science 
teaching materials, and will offer opportunities 
for the implementation of new methods of teaching 
science developed elsewhere, and will permit 
departments and institutions to develop their own 
approaches to science 'teaching . CAUSE will also 
provide funds for the further training of science 
faculty and for the addition of new science 
faculty. Grants will be made on a competitive 
basis . . . Interdisciplinary approaches to 
teaching of science will be encouraged. Parti- 
cular emphasis will be placed on encouraging the 
teaching of science in minority institutions and 
in two-year colleges. 

In the final NSF Appropriations Act of 1976, $2.3 million out of a 
total budget of $10 million was set aside for 2-year institutions. 

The Conference Report does not explicitly define the term 
"comprehensive". NSF has offered this definition in contrast to 
its other programs: 

- . . CAUSE, however, offers a comprehensive 
approach to the improvement of an institution's 
instructional programs by support of broader, 
more integrated projects. Therefore, a CAUSE 
proposal must be built upon the application of 
a set of coordinated activities . . . These 
activities are expected to improve science 
education in a particular department or a larger 
unit within the institution. (CAUSE Program 
Announcement, 1978) 

In summary, the Congressional intent in establishing the 
CAUSE program was to: (1) strengthen undergraduate science educa- 
tion; (2) provide integrated and broadly-based science improvement 
programs at the institutional level; (3) provide for faculty 
development; (4) encourage interdisciplinary approaches to the 
improvement of science programs; (5) place particular emphasis on 
improvement of science education at minority and 2-year insti- 
tutions. 



Description of the Evaluation 



In the summer of 1977, the Office of Program Integration at 
the National Science Foun-dation invited interested organizations 
to submit proposals for the evaluation of the CAUSE program among 
others. The general purpose of the evaluation, according to NSF's 
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original solicitation, was to elicit information about the needs, 
implementation strategies, outcomes and potential program modifi- 
cations as these were relevant at the institutional level. The 
evaluation was not to focus on NSF's internal decision-making, 
policies, procedures or selection of awards since these were the 
purview of external oversight activities. 

In responding to the needs and priorities identified by NSF , 
Development and Evaluation Associates, Inc., designed an evaluation 
which would: (1) be responsive to the wide diversity in CAUSE 
projects at the institutional level; (2) be formative in nature, 
facilitating the refinement and improvement of the CAUSE program; 
(3) be comprehensive in its scope and at the same time provide 
detailed examination of a few CAUSE projects. 

Out of the issues identified by NSF, we formulated six which 
we believed would potentially yield the most useful information: 

1. To what extent are high priority institutional needs 
being met by local CAUSE projects? 

2. How are the CAUSE projects being implemented? What are 
their strengths and weaknesses? 

3. To what extent is the improvement of the quality of 
instruction occurring as a result of CAUSE? 

4. What is the nature and quality of the evidence and the 
evidence collection and analysis procedures being used 
to determine the strengths and weaknesses of individual 
CAUSE projects? 

5. What are the relative costs of the design, implementation, 
and operation of activities within CAUSE projects, and 
how do they relate to post-CAUSE institutional support? 

6. What policy changes and recommendations should be 
implemented in the CAUSE program? 

In seeking ways to be both comprehensive and provide detailed 
examinations of some CAUSE projects, we designed a three- pronged 
evaluation (see Figure 1). 

The broad focus evaluation . This component of the evaluation 
provided information on all CAUSE projects funded between 1976 and 
1978. It included an examination of proposals submitted to NSF 
and subsequently funded and a survey of all project directors. 

The medium focus evaluation . This component consisted of 
one-time only site visits to 17 CAUSE projects in order to pro- 
vide a look at the projects from the participants' point of view 
and provide some insight on how CAUSE projects were being imple- 
mented. 
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The narrow focus evaluation . This component of the evaluation 
provided an in-depth view of eight CAUSE projects with multiple 
site visits to each. A detailed understanding of these projects 
was made possible by this approach. The cost issue was explored 
in connection with these case studies. 



A SUMMARY OF THE CONCLUSIONS 



Introduction 



Chapter Three of Volume I of the evaluation report presents 
the evaluation's overall conclusions. The conclusions are 
organized around the issues whose choice and definition are 
explained in Chapter Two. Various conclusions are presented rele- 
vant to each issue. An explanation of each conclusion is provided 
along with a series of references to some of the specific evalu- 
ation findings which led to the conclusion. The reader is 
encouraged to use these references as a bridge to the more detailed 
findings and evidence presented in Volumes II and III. 

The contents of Chapter Three represent the ultimate level of 
refinement for the evidence gathered through the evaluation's 
various activities. Other analyses of findings are presented in. 
the first chapters of Volumes II and III, but these analyses are 
solely at the level of the study's narrow and medium, or broad 
focus methods respectively. The conclusions presented here and 
in Chapter Three are based on all aspects of the evaluation. 



The Concl us i ons 



The conclusions are organized around the first five issues of 
tLe evaluation. Issue Six, relating to recommendations for NSF 
policy, is discussed separately in Volume I and in this executive 
summary- 

The following are the major conclusions of the evaluation of 
CAUSE. Fuller explanation and discussion of each conclusion are 
available in Chapter Three of Volume I. 

Conclusion 1.1: Most CAUSE projects would not have occurred with - 
out CAUSE funding . 

This is perhaps the most basic conclusion of the evaluation. 
It confirms the fact that the NSF funds are necessary to establish 
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projects of the type sponsored by CAUSE. (This, of course, implies 
nothing about the importance or appropriateness of the projects 
themselves, which will be dealt with in later conclusions.) 

Approximately half of the project directors surveyed and 
visited reported that they would have "given up on their projects 
for lack of funds" if CAUSE money had not been available. The 
reports of the site visitors confirm this perception. It appears, 
further, that those who would have proceeded without CAUSE funds 
would have had to do so at an extremely reduced capacity and in 
almost all cases with a drastically different project. 

Conclusion 1.2: CAUSE Projects Fill Needs Perceived as Important 
within Institutions which Receive the Grants 

The CAUSE program stresses a focus on locally defined needs 
as opposed to more general, nationally defined needs. The hope 
expressed in the legislation is that the CAUSE program M . . . will 
permit departments and institutions to develop their own approaches 
to science teaching. 11 The CAUSE program guidelines require a 
statement from the institutional administration .(usual ly sijned by 
the president or academic vice-president) describing the project's 
relationship to institutional goals and stating the institution's 
commitment to and support of the project's implementation and 
post-grant continuation. This description and justification of 
local needs is seen as one means of insuring the necessary insti- 
tutional support for project implementation and the eventual con- 
tinuation of project improvements in post-grant years. 

Conclusion 2.1: CAUSE Projects are Being Implemented as Proposed . 

The relationship between the project as described in the pro- 
posal and the project as implemented generally was an initial 
focus of the site visitors, not as an assessment of compliance but 
as a means of becoming quickly acquainted with the nature of the 
ongoing project. The survey of project directors also asked 
several questions on this point. In the opinion of the project 
directors as well as the site visitors, there was a very close 
match between projects as proposed and projects as implemented. 

This generalization, however, should be qualified on two 
counts. First, there frequently were some variations from original 
plans, usually in terms of emphasis and occasionally through the 
dropping of one or two original objectives. Sometimes this was 
done through a formal amendment to the grant, but more often it 
simply occurred during project implementation. The variations 
were rarely significant in cerms of the project's overall goals 
and often represented reasonable judgments on the part of the pro- 
ject director. The need far modifications usually resulted either 
from an initial lack of understand!' ng of the resources necessary 
to carry out the proposed tasks (this was especially true with 
respect to materials and curriculum design activities) or from a 
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mid-project decision to de-emphasize the project's relatively 
unsuccessful aspects and emphasize its more successful aspects. 
Occasionally changes had to be made due to changes in project 
personnel . 

Conclusion 2.2: CAUSE Projects Serve as Catalysts in Focusim, 
Motivating and Coordinating Instructional Improvemen t Activities 
of Science Facul ty , 

High guality projects are characterized by high guality pro- 
ject staff. While these people generally were present at the 
institution before the award of the grant, due to various institu- 
tional cons trai nts. they may not have worked together previously 
on a major instructional improvement effort. The CAUSE project 
acts as a catalyst and overcomes some of these real or perceived 
constraints and brings these people together. Typical constraints 
which CAUSE funding can help to overcome include lack of time, 
opportunity (and often precedent) to work with people other than 
immediate colleagues, and a general lack of substantive support 
for the improvement of instruction within the institution. 

Conclusion 2.3: Projects Are Generally Adequately Managed, but 
Project Directors Often Do Not Have Access to Areas of Expertise 
Important to Project Implementation . 

While it was found that most projects are achieving (sometimes 
surpassing) their proposed objectives, it also is the case that 
these achievements are frequently limited by or require more effort 
than necessary due to a lack of the required expertise. Generally, 
this lack of expertise is not in the area of project management 
since project directors tend to be experienced faculty who have the 
respect of their colleagues and are effective leaders. Instead, 
the lack of expertise generally involves some specific activity of 
the project itself. Such deficiencies were often recognized in the 
proposal and plans were made to compensate for the missing exper- 
tise by hiring consultants. Sometimes, however, the need for 
certain kinds of expertise had not been adequately recognized in 
planning the project. 

Conclusion 2.4: Effective Project Directors Tend to Possess 
Several Common Characteristics . 

In most cases, project directors who are effective are senior 
faculty members and had primary responsibility for writing the 
proposal. Their involvement in the proposal writing meant that 
they were the ones primarily responsible for formulating project 
objectives and implementation plans. Thus they had thorough know- 
ledge and understanding of the rationale of project activities 
from the inception of project implementation. In most successful 
projects which span departments or divisions of the institution, 
the project directors hold administrative positions such as 
department chairperson. Presumably, senior faculty are more able 
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than junior faculty to weather the political implications of 
administering across organizational units and are in addition less 
susceptible to "publish or perish 11 pressures. 

Good project directors provide strong personal leadership and 
are innovators at their institutions. Generally they are among 
the first at their institutions to urge adoption of local innova- 
tive activities, typically in the guise of a CAUSE project. Most 
ar: w<±ll respected among colleagues and administrators and set 
examples of hard work and enthusiasm for the project. A related 
characteristic is. that effective project directors are frequently 
adept at handling institutional politics. This appears to be most 
important when projects cross organizational lines within the 
institution. 

Conclusion 2.5: The Use of Release Time Is a Critical Factor in 
the Efficient and Effective Use of CAUSE Dollars . 

The use of release time appears to be of extreme importance 
in the successful completion of CAUSE projects. In implementing 
the projects, however, it is sometimes very difficult to allocate 
or administer release time effectively. Problems involving the 
effective allocation of release time generally stem from a number 
of sources including the nature and size of the institution, the 
accuracy of the planned-for level of effort, institutional policies, 
and the willingness of institutional administrators to live up to 
commitments made. For some projects, the timing of release time 
and the intensity of project activities proved to be a problem. 

Conclusion 2.6: Institutional Support Is Critical to Successful 
Project Imp 1 emen ta ti on 

Horizontal and vertical support for CAUSE projects appears to 
be generally adequate for project success. However, there does 
seem to be definite room for improvement, and for some projects 
lack of support is a serious problem. This lack of support is 
sometimes due to a lack of skills, abilities, and political power 
of the project director and project faculty. At other times it is 
due to a lack of a clear understanding and definition of what 
institutional needs are being served by the project and why they 
are of high priority. 

Conclusion 2.7: CAUSE Projects Often Represent Innovations Within 
Their Respective Institutions Which Are Not Usually Innovations to 
Science Education . 

The large majority of project directors (943) responding to 
the first survey reported that their projects were either very 
innovative (52%) or somewhat innovative (425S) as compared to the 
regular activities of the departments ) involved in CAUSE. Obser- 
vations of the site visitors confirmed this report. Project 
directors are in general viewed as innovators within their depart- 
ments and most often the project activities are unlike activities 
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occurring in related departments. 

In general, however, the projects cannot be characterized as 
innovative with respect to science education as a whole since most 
of the projects we visited had counterparts occurring somewhere 
else in the country under the auspices of the CAUSE program. In 
fact, there are strong similarities among projects even within the 
sample of 25 we observed. Although certain aspects of some of the 
projects do represent a unique approach or a unique application of 
a certain technology,' there tends to be a substantial overlap among 
projects with respect to the problems grappled with and in the 
solutions found. 

It is not necessa ry to be innovative nationally to be success- 
ful . It is not even necessary to be innovative locally if the 
needs are legitimate and of high priority. It is important to 
recognize, however, that many projects that are innovative locally 
could benefit substantially from investigating the present and 
previous efforts of others attempting similar projects so that 
something might be learned from the successes and failures of 
others . 

Conclusion 2.8: Comprehensive Projects Have Stronger Potential 

for Overall Lasting Improvements to Undergraduate Science Education , 

In reviewing project implementation and impact at the 25 pro- 
jects we visited, one overall theme emerged: the best projects 
were those which were most comprehensive. The CAUSE program 
guidelines encourage the conduct of comprehensive projects. Such 
projects are described as "those which are broader and more inte- 
grated,, and which include a set of coordinated acti vi t i es 11 . 

Conclusion 3.1: Science Education Resources Have Seen Improved . 

CAUSE projects generally have strengthened science education 
resources with respect to instructional materials, laboratory and 
instructional equipment, instructional facilities, and the develop- 
ment of faculty skills in i nstructi onal ly related areas. The 
improvements wrought by the projects are likely to provide in most 
cases, long-term benefits to institutions in terms of their capa- 
bility to provide quality science programs. Particularly in those 
projects which have involved the acquisition of equi pment and/ or 
the acquisition/renovation of institutional facilities, more or 
less permanent improvement in science education resources have 
been made. Of course, there are exceptions, especially in those 
cases where project activities have primarily consisted of "one 
time" occurrences and where there appears to be little institu- 
tional support for th* continued pursuit of project goals. 
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Conclusion 3.2: Institutional Capabilities for Sel f- Assessment , 
Management and Evaluation Have Not Been Substantially Changed . 

One of the primary charges of the CAUSE program is to improve 
the capabilities of institutions to perform self-assessment, 
management, and evaluation. On the basis of the evidence collected 
during the course of this study, however, it appears that this goal 
has fallen short of the mark. This is not to say that the CAUSE 
projects have not had some impact on the capabilities of individuals 
within the institutions with respect to these areas. Indeed, many 
projects have provided a powerful means of professional development 
for faculty (especially project directors), particularly in the 
area of management. 

Conclusion 3.3: The Quality of Science Instruction Has Been 
Improved Within Most Institutions Funded by CAUSE 

The strongest evidence for the improvement of the quality of 
instruction is the expansion of instructional opportunities for 
students. Many of these opportunities have been designed into 
courses as required activities and many others have been designed 
to be supplementary to existing course materials and used at the 
students' option. In many c?ses the CAUSE projects have made 
available educational experiences that simply were not available 
previously (e.g., through the acquisition and development of 
laboratory facilities, both self-study and traditional, computer 
hardware and software, revised curricula and the like). Although 
instructional materials prepared under CAUSE grants vary widely 
in terms of normal instructional design criteria (clarity, organi- 
zation, specification of goals and objectives, readability, appro- 
priate use of the medium, etc.), most are being used by faculty 
and students who generally report them to be useful. 

Conclusion 3.4: The Direct Impact of CAUSE Projects on Students 
Cannot Generally Be Documented . 

Very little evidence exists at most CAUSE projects which shows 
a direct impact of project activities on students 1 knowledge, 
attitudes and behaviors. When such evidence does exist, as in the 
form of test scores or feedback on student questionnaires, it is 
difficult if not. impossi ble to determine from available data 
whether or not the impact is more, less or different from impacts 
that could have been expected from participation in non-CAUSE or 
pre- CAUSE educational experiences. 

Conclusion 3.5: A Range of Secondary and Unintended Effects Have 
Occurred as a Result of the CAUSE Projects . 

Several outcomes of the CAUSE projects have occurred which 
might best be classified as secondary or unintended since they are 
not within the scope of the original intentions or goals of the 
project. Nevertheless, these effects probably have important 
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implications for the quality of the science programs at the insti- 
tutions at which they have occurred. 

Conclusion 4.1: CAUSE Has Resulted in Increased Awareness of 
Evaluation in Undergraduate Science Education . 

A major goal of the CAUSE program is to "increase institutions' 
capability for self-assessment. . . and evaluation of their science 
programs". The present study found that the CAUSE program has 
resulted in an increased awareness of the role of evaluation in 
improving undergrate science education programs. While an increased 
awareness is short of an actual increase in capability and appli- 
cation, increased awareness is an important initial step to 
increased capability and usage. 

Conclusion 4.2: There Is Widespread Uncertainty Among Project 
Staff About the Nature and Purpose of Evaluation . 

While many project directors, in fact the vast ma j or i ty , . agree 
with statements from the survey like, "It is important that CAUSE 
guidelines require evaluation as part of projects", and "Evaluation 
is important at our institution in monitoring the effectiveness of 
projects of this type", there is also a strongly held feeling that 
precious project resources should not be used for evaluation when 
they are needed for implementation activities. The agreement with 
statements affirming the importance of evaluation suggests that 
some project directors are likely to verbally support evaluation 
while also feeling that evaluation is the least important of all 
project priorities. Consequently, there is a lack of clarity and 
focus in evaluation activities at CAUSE institutions. Sometimes 
the evaluation proposed initially has not been done or has been 
modified. Sometimes data are not utilized effectively. 

Conclusion 4.3: Evaluations as Implemented Vary Considerably in 
Substance from Evaluations as Proposed . 

The majority of project directors responding to the survey 
reported that evaluation plans were being carried out as proposed. 
However, survey results also showed evaluation to be one of the 
activities which, in practice, most frequently deviated from the 
proposal's plans. This finding was generally confirmed during the 
site visits: evaluation plans were often being followed in name 
only with relatively little attention. to the role evaluation acti- 
vities could play in conducting or improving project activities. 

Conclusion 4.4: CAUSE Has Had Little Impact on Institutions' 
Capabilities for Self-Assessment . 

Because evaluation has not played an important or prominent 
role in the individial projects and has had little effect on them, 
it has not been possible for the CAUSE projects to have any real 
impact on an institution's capability for self-assessment or 
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evaluation of their science education programs. The project 
directors who already knew something about evaluation pretty much 
administered the evaluation activities themselves, resulting in no 
institutional change. Those who knew little about evaluation 
either retained the services of an evaluation consultant or an 
expert from a relevant science discipline from within the institu- 
tion or from some other institution. In either of these cases 
very little interaction occurred between evaluator and project 
staff which would increase the staff's capabilities in evaluation. 
In the case of evaluation consultants, efforts were focused on the 
collection of data. In the case of discipline experts, efforts 
focused on a review of course content. 

Conclusion 4.5; Evaluation Has the Potential to Improve CAUSE 
Projects in Important Ways . 

The improvement-oriented or formative role of evaluation has 
the most potential for improving projects in that it relies 
heavily on the already existing expertise of project directors and 
staff and can have a direct impact on the achievement of their own 
objectives. While there are specific data collection and analysis 
techniques useful to formative evaluation, we saw several instances 
in which project staff with no training in evaluation were con- 
ducting extremely useful formative evaluations. The evaluation 
activities, sometimes not labeled as such, were characterized by 
explicit consideration of desired outcomes and careful observation 
of interim products (and the initial use of these products) to 
determine the extent to which the outcomes were being achieved. 
Sometimes these observations were formal and sometimes they were 
informal. However, they were always conscious of and directed 
toward collecting information useful to the improvement of project 
activities or outcomes. 

Conclusion 5.1: Most CAUSE Projects Support Design and Implemen - 
tation Expenditures for Instructional Improvements with Low 
Recurring Costs for Post-Grant Continuation of the Improvements . 

Overall we found that CAUSE funds generally have supported 
the design and investment expenditures necessary for capital 
improvement projects. Whether they have been primarily instruc- 
tional development efforts or additions to equipment or facilities, 
most projects have represented a one-time allocation of resources 
to accomplish a specific kind of improvement that would continue 
to provide benefits in the future with little or no marginal 
recurring costs for their operation. 

Conclusion 5.2: Institutional and Personal Contributions to CAUSE 
Projects Almost Always Exceed the Contributions Originally Proposed . 

In most cases the local CAUSE projects have served to coalesce 
and focus local resources for the improvement of science education 
beyond the extent originally proposed. This was observed and 
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documented most clearly in the cost analyses at the institutions 
described in the eight case studies, but it was frequently noted 
by site visitors to the remaining 17 institutions as well. Some- 
times the additional institutional contributions have been quite 
obvious as when the institution covered the extra cost of addi- 
tional or higher quality equipment than originally proposed. At 
other times these additional contributions have been less obvious, 
■although just as real, as when the presence of the project has 
served as a means of attracting a variety of other institutional 
♦resources in the form of donated administrative time, or the use 
of additional university facilities or services for project- 
related purposes. The increase in visibility and reputation of a 
given department has also sometimes been reported as providing 
additional leverage to local science departments during institu- 
tional budget negotiations. Substantial personnel resources have 
also been donated to the projects, most often by individual pro- 
ject faculty, in the form of evening and weekend hours. 



RECOMMENDATIONS 



Introduction 



A question asked of project directors and staff during the 
course of the various site visits and also on the two surveys was 
"If you had one thing to recommend to the National Science Founda- 
tion regarding the CAUSE program, what would it be?". This ques- 
tion was often asked among the site visitors themselves during the 
course of the evaluation, and was asked again in writing of each 
of the site visitors before final deliberations regarding the con- 
clusions and recommendations were conducted. Many more recommenda- 
tions have been suggested over the course of the evaluation, most 
of which have been discussed or at least implied in the various 
levels of analysis included in Volumes II and III of this report. 
The recommendations presented here are those which emerged most 
frequently during these discussions and which appear to have the 
greatest potential for strengthening the CAUSE program. 

Recommendation One: The CAUSE Program Should Be Continued and 
Strengthened . 

The CAUSE program has been effective in improving the nation's 
science education resources. Institutional needs for improvement 
of instruction in science education are real and institutional 
budgets are unable to support the initiation of the sort of com- 
prehensive projects brought about by the CAUSE program. Projects 
have been effectively implemented and are meeting institutional 
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needs as perceived by local science faculties. Through the CAUSE 
program's requirement of matching funds as well as through the 
catalytic effect of many funded projects within their respective 
institutions, the CAUSE program has encouraged the targeting of 
institutional and human resources toward the improvement of under- 
graduate science education. The CAUSE program is effective. 

Although the basic organization and intent of the CAUSE pro- 
gram is sound and no significant weaknesses have been identified 
as inherent to its general approach, we believe that the CAUSE 
program can be strengthened significantly through an increase in 
emphasis in a number of critical areas. 

Xr, 

Recommendation Two: T he CAUSE Program Staff Should Be Enlarged, 
or Other Means of Providing Additional Personnel Support to the 
CAUSE Program Should be Provided . 

There were very few complaints voiced ,by project directors 
about NSF's administration of the CAUSE program. Quite the con- 
trary, comments on the surveys as well as those made during site 
visits were quite positive. Although contact between CAUSE pro- 
gram staff and individual projects was infrequent, the contacts 
that did occur were well regarded. 

However, it was clear from observations made during the site 
visits that a number of problems could have been avoided and pro- 
jects often could have 'been more effectively implemented had the 
expertise of fellow scientists experienced in the improvement of 
instruction through large-scale development efforts been available. 
We feel the advice and support that is presently being given to 
project directors by program staff is appropriate and useful and 
that an increase of such advice and support would result in a more 
effective CAUSE program. 

Recommendation Three: Project Directors Should Be Encouraged and 
Assisted in Obtaining Specialized Expertise When Needed . 

There are a number of specific areas of expert ise useful to 
directors of large-scale instructional improvement projects. Com- 
ments made by project staff on survey forms and during the course 
of site visits as well as observations made by the site visitors 
themselves indicate that expertise in many areas is often unavail- 
able to given projects. Specific types of expertise noted to be 
lacking most often include expertise in evaluation, instructional 
development, project management, hardware acquisition and instruc- 
tional computer programming. 

Possible ways in which obtaining assistance in these areas 
could be encouraged, when the need exists, include suggesting the 
use of outside experts through program guidelines or during 
negotiations, supplying project directors with lists of sources of 
available expertise in each of these areas, and sponsoring targeted 
workshops for project directors needing assistance. 
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Recommendation Four; Communications and Collaboration Across the 
Country Among Project Staff and Others Involved in and/or Experi - 
enced with Similar Instructional Improvement Efforts Should Be 
Encouraged and Supported . 

Improvements in the communication and collaboration between 
CAUSE projects could greatly improve the efficiency and effective- 
ness of many projects. Among the 25 projects visited as part of 
this study, many were quite similar in nature and intent. Many 
more similarities undoubtedly exist among all CAUSE-funded projects, 
not to mention other instructional improvement efforts not funded 
by CAUSE. As the site visitors went from project to project, they 
frequently saw project staff addressing and solving many of the 
same problems, but usually q/ily after considerable time and effort 
had been expended. In those few projects where project staff had 
access to similar projects elsewhere, they seemed to benefit 
greatly from the experience of others. Unfortunately, most pro- 
ject directors of other projects similar to their own have little 
means of access to those projects, or do not understand how their 
project might benefit from another's experience. 

This situation exists in spite of efforts by the CAUSE pro- 
gram to disseminate information on funded projects and to bring 
the project directors together to meet one another. While these 
efforts are appropriate, they are insufficient. The kind of infor- 
mation that nrost project directors would benefit from has more to 
do with the process of conducting project activities than with the 
products of those activities. New project directors need to hear 
the stories of how other projects have been conducted. They need 
to be able to talk to someone about the minutiae of project 
activities. 

It seems clear that the more that can be done to foster com- 
munication and collaboration among CAUSE project directors the 
better. In addition to the CAUSE program's existing efforts in 
this area, it is recommended that CAUSE encourage or foster further 
opportunities for project directors to meet and talk with one 
another. Specifically, the role of the present project directors' 
meeting should be clarified and extended. All project directors 
should attend one such meeting a year. This meeting should be 
longer than it is at present, provide more opportunity for project 
director to project director interaction, and (to accommodate the 
needs of new projects) should be held in the summer rather than in 
the fall. The conduct of such a meeting could represent an 
unrealistic burden for CAUSE program staff; however, the meeting 
could be organized and largely conducted by any of a number of 
external groups or organizations familiar with the CAUSE program. 
Much of the cost of these meetings would be underwritten by stipu- 
lated allocations in project budgets. Furthermore, it is recom- 
mended that proposal guidelines stress the desirability of estab- 
lishing budget allocations for other kinds of project-related 
professional development activities. In addition to the project 
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director meetings, most projects would benefit from opportunities 
for their faculty to attend workshops, professional association 
meetings and other events related to their projects' activities. 

A number of other steps could be taken as well to further 
project directors' access to information on the process of con- 
ducting CAUSE projects. Careful descriptions of projects, such 
as the case studies presented in Volume II of this report, should 
be disseminated to the project directors. Proposals, interim 
reports and evaluation findings of selected projects could be 
gathered together in a kind of lending library for project direc- 
tors. (It would also clarify the role of project interim reports 
and final reports if project directors understood that these 
reports were being written to inform future project directors.) 

Other ideas along these lines can be generated and these need 
more refinement. The important thing to understand is that to be 
successful most CAUSE projects require some professional develop- 
ment of project staff in such areas as instructional development, 
instructional technology, and so on. The CAUSE program should 
devote further attention to facilitating the professional develop- 
ment aspects of the projects in order to maximize their efficiency 
and effectiveness. 

Recomme ndation Five: A Pro ject Pi rectors ' Reference Manual Should 
Be Crea ted to Provide Support and Direction to Project Staff, Par - 
ticularl y in the Areas of Project Management, Evaluatio n and 
Instructional Design . ; 

There are a sufficient number of problems faced in common by 
most project directors and staff to justify the creation of a 
reference manual describing these potential problems and alternate 
means of attacking them. Included in such a manual would be 
descriptions of various management, evaluation and instructional 
design strategies used by successful project directors, and cri- 
teria to aid in the selection of the appropriate strategy for a 
given situation. Of particular use would be hints, guidelines, 
and forms useful for the monitoring and control of project activi- 
ties. The selection and use of outside experts would also be 
covered, as would be hints on the successful use of formative 
evaluation strategies. References to additional sources of infor- 
mation would also be included. The availability of such a manual, 
which would rely primarily on the experience of project directors 
of past CAUSE programs, would greatly decrease the present prob- 
lems due to each project tending to start anew with little aware- 
ness of the progress already made by others. 

Recommendation Six: The CAUSE Program's Present Emphasis on Local 
Need and Institutional Support Should be Continued, If Not Increased . 

Effective CAUSE projects are based on hign priority local 
needs. Although this statement is somewhat obvious, its implica- 
tions for the planning and implementing of projects are not. It 
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is relatively easy to construct a statement of institutional need 
that does not contradict the institution's general goals and which 
directly supports whatever instructional improvement project a 
creative faculty member would like to implement. However, it is 
extremely difficult to identify, clarify and articulate a con- 
sensus on local instructional needs in a way that drives and 
focuses project planning, design, implementation and evaluation. 
However, without a clear, common and meaningful understanding of 
what needs are to be filled and exactly how project activities 
are to meet those needs, the filling of the needs will depend more 
on chance than on design. Because it is so easy for institutional 
administration and project staff to climb aboard the bandwagon of 
an intuitively appealing solution and so difficult to clearly and 
meaningfully articulate the concept of high priority institutional 
need, it is critical that project staff be encouraged and assisted 
with respect to recognizing the importance of local institutional 
needs as the driving force of their projects. 

A number of specific actions can be taken to make the present 
emphasis on institutional needs more meaningful to project planners 
and implementers . The statement of institutional support should 
represent the commitment not only of the institution's highest 
levels of administration, as is presently the case, but of the 
department chair(s) and dean(s) levels as well. This statement of 
commi tment. shoul d explicitly define the specific needs to be 
addressed by the project and should describe the institution's 
specific commitment to the project in terms of how faculty time 
will be reallocated, what responsibilities participating faculty 
will be released from (and how those responsibilities will be 
covered), and the incentives the institution will provide partici- 
pating faculty to enhance their continued interest and participa- 
tion in the project. Evidence of individual faculty members' per- 
sonal and professional commitment to the project should also be 
emphasized in project proposals. 

Recommendation Seven: A Greater Emphasis Must Be Placed on the 
Clarification and Improvement of Proposals Once the Proposal Review 
Process Is Completed but Before the Grants Are Formally Awarded . 

Some amount of negotiation between CAUSE program staff and 
selected grantees presently occurs as part of the formal award 
procedure and the modifications to original proposals recommended 
by proposal reviewers and CAUSE staff appear to have been generally 
sound. (Although pre-award negotiations were not a formal focus of 
our evaluation, documentation of these nego ti a t i ons was often 
reviewed by site visitors.) However, we believe that it is at this 
point in the project development process that NSF has the greatest 
power to positively effect the quality of CAUSE projects. Project 
staff will be more strongly motivated to put additional time and 
effort into the improvement of proposals because they know the 
probability of award is great. This is also an opportune time to 
require the institution's administrators and department faculty to 
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become more precise about the nature of their commitment. In par- 
ticular, it would be an ideal time to fully explicate the issues 
of release time, management and logistics, and evaluation. 

Since the recommendations of the proposal review teams are an 
important source of information for CAUSE program staff during the 
pre-award process, it is important that each review team include 
at least one person knowledgeable and experienced in comprehensive 
instructional improvement efforts in higher education. We believe 
a number of.problems within specific projects could have been 
avoided had they been identified during the proposal review process. 

Recommendation Eight: More Emphasis Should Be Placed on the Rela - 
tionship Between Development and Operation Costs Within Proposed 
Projects . 

The distinction between development costs (i.e., the costs 
required to bring about an instructional improvement) and recurring 
operation costs (i.e., the costs required to maintain an instruc- 
tional improvement) is not usually addressed in project proposals, 
nor does it appear to be one consciously made by many project 
directors. However, the formal consideration of the cost struc- 
tures of the eight case study sites as well as a more informal con- 
sideration of costs within each of the other sites visited strongly 
suggests that the distinction is extremely important in CAUSE, par-, 
ticularly with respect to its bearing on the post-grant maintenance 
of projects. The costs of most of the projects visited were pri- 
marily development costs. We believe that most of the instructional 
improvements brought about by these projects will continue to be 
maintained by their institutions upon completion of the grant. The 
relatively high recurring operation costs of a few of the projects 
raised some doubts as to these projects 1 continued viability within 
their respective institutions in post-grant years. During the 
solicitation and award process an explicit emphasis should be 
placed on determining the proportion of the project which is a re- 
curring operation cost* Those projects with a high portion of 
recurring operation costs should be required to carefully justify 
the projects 1 plans in terms of long-term benefit to the institu- 
tion and/or maintenance of the projects 1 improvements. The com- 
mitment and ability of the institution to continue the proposed 
instructional improvement in post-grant years should be examined 
very closely by the proposal reviewers and the CAUSE program staff. 



Summary 

The CAUSE program is meeting extremely important institutional 
needs for improved science education resources. Some strengthen ing 
of the program, as suggested by the above recommendations, would 
enhance the implementation and increase the impact of local projects. 
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